Tiny Machine Learning Environment: Enabling Intelligence on Constrained Devices

Running machine learning algorithms (ML) on constrained devices at the extreme edge of the network is problematic due to the computational overhead of ML algorithms, available resources on the embedded platform, and application budget (i.e., real-time requirements, power constraints, etc.). This required the development of specific solutions and development tools for what is now referred to as TinyML. In this dissertation, we focus on improving the deployment and performance of TinyML applications, taking into consideration the aforementioned challenges, especially memory requirements. This dissertation contributed to the construction of the Edge Learning Machine environment (ELM), a platform-independent open source framework that provides three main TinyML services, namely shallow ML, self-supervised ML, and binary deep learning on constrained devices. In this context, this work includes the following steps, which are reflected in the thesis structure. First, we present the performance analysis of state of the art shallow ML algorithms including dense neural networks, implemented on mainstream microcontrollers. The comprehensive analysis in terms of algorithms, hardware platforms, datasets, pre-processing techniques, and configurations shows similar performance results compared to a desktop machine and highlights the impact of these factors on overall performance. Second, despite the assumption that TinyML only permits models inference provided by the scarcity of resources, we have gone a step further and enabled self-supervised on-device training on microcontrollers and tiny IoT devices by developing the Autonomous Edge Pipeline (AEP) system. AEP achieves comparable accuracy compared to the typical TinyML paradigm, i.e., models trained on resource-abundant devices and then deployed on microcontrollers. Next, we present the development of a memory allocation strategy for convolutional neural networks (CNNs) layers, that optimizes memory requirements. This approach reduces the memory footprint without affecting accuracy nor latency. Moreover, e-skin systems share the main requirements of the TinyML fields: enabling intelligence with low memory, low power consumption, and low latency. Therefore, we designed an efficient Tiny CNN architecture for e-skin applications. The architecture leverages the memory allocation strategy presented earlier and provides better performance than existing solutions. A major contribution of the thesis is given by CBin-NN, a library of functions for implementing extremely efficient binary neural networks on constrained devices. The library outperforms state of the art NN deployment solutions by drastically reducing memory footprint and inference latency. All the solutions proposed in this thesis have been implemented on representative devices and tested in relevant applications, of which results are reported and discussed. The ELM framework is open source, and this work is clearly becoming a useful, versatile toolkit for the IoT and TinyML research and development community

Study on the effect of wormseed plants; artemisia cina L. and chamomile; matricaria chamomilla L. on Growth Parameters and Immune Response of African Catfish, Clarias gariepinus

A number of 420 fingerlings of catfish were used to examine the effect of wormseed plants and chamomile on the growth parameters and on non-specific immune response of the African catfish; Clarias gariepinus. Both types of herbs were used in rates of 1, 3 and 5% with 3 replicates per each of the 6 treatments. The 7th treatment was kept as a control group. The experimented catfish were fed with the 7 examined diets in the rate of 3% of fish biomass for 1 month. Different growth parameters as well as blood parameters were estimated to evaluate the growth performance and immune response of the experimented catfish. Results revealed that wormseed plants Artemisia cina L. in the rate of 3 and 5% and chamomile Matricaria chamomilla. L. in the rate of 1% showed the best figures of growth parameters as well as immune response parameters of the examined catfish

Analysis, Optimization & Execution of General Purpose Multimedia Applications on subword VLIW Datapaths

To be added later..

Underlying collegial relationships controlling project implementation : case study in Egypt

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Urban Studies and Planning, 1990.Includes bibliographical references (leaves 109-130).by Hoda A.F. Tolba Sakr.Ph.D

Pauli spin susceptibility of a strongly correlated two-dimensional electron liquid

Thermodynamic measurements reveal that the Pauli spin susceptibility of strongly correlated two-dimensional electrons in silicon grows critically at low electron densities - behavior that is characteristic of the existence of a phase transition.Comment: As publishe

Survival or Sustainability? Contributions of Innovatively-Managed News Ventures to the Future of Egyptian Journalism

In the repressive political climate prevailing in Egypt in 2013-15, news ventures aspiring to high standards of reporting were forced to innovate. This paper analyses three Egyptian start-ups that experimented with novel revenue streams and news services during that period, to gain insights into their approaches to managing journalism. In the process it compares different criteria for assessing sustainability and concludes that, in adverse political environments, narrow economic measures of profitability and survival may give a misleading picture as to the sustainability of the kind of journalism conducive to democratic practice. Operating collaboratively, transparently and ethically may slow productivity and profitability in the short term while laying stronger foundations for durable relations among media teams, as well as with readers and advertisers, in the long run

Impact of survey geometry and super-sample covariance on future photometric galaxy surveys

Photometric galaxy surveys probe the late-time Universe where the density field is highly non-Gaussian. A consequence is the emergence of the super-sample covariance (SSC), a non-Gaussian covariance term that is sensitive to fluctuations on scales larger than the survey window. In this work, we study the impact of the survey geometry on the SSC and, subsequently, on cosmological parameter inference. We devise a fast SSC approximation that accounts for the survey geometry and compare its performance to the common approximation of rescaling the results by the fraction of the sky covered by the survey, fSKY, dubbed ‘full-sky approximation’. To gauge the impact of our new SSC recipe, that we call ‘partial-sky’, we perform Fisher forecasts on the parameters of the (w0, wa)-CDM model in a 3 × 2 point analysis, varying the survey area, the geometry of the mask, and the galaxy distribution inside our redshift bins. The differences in the marginalised forecast errors –with the full-sky approximation performing poorly for small survey areas but excellently for stage-IV-like areas– are found to be absorbed by the marginalisation on galaxy bias nuisance parameters. For large survey areas, the unmarginalised errors are underestimated by about 10% for all probes considered. This is a hint that, even for stage-IV-like surveys, the partial-sky method introduced in this work will be necessary if tight priors are applied on these nuisance parameters. We make the partial-sky method public with a new release of the public code PySSC

Quantifying Demonstration Quality for Robot Learning and Generalization

Learning from Demonstration (LfD) seeks to democratize robotics by enabling diverse end-users to teach robots to perform a task by providing demonstrations. However, most LfD techniques assume users provide optimal demonstrations. This is not always the case in real applications where users are likely to provide demonstrations of varying quality, that may change with expertise and other factors. Demonstration quality plays a crucial role in robot learning and generalization. Hence, it is important to quantify the quality of the provided demonstrations before using them for robot learning. In this paper, we propose quantifying the quality of the demonstrations based on how well they perform in the learned task. We hypothesize that task performance can give an indication of the generalization performance on similar tasks. The proposed approach is validated in a user study (N = 27). Users with different robotics expertise levels were recruited to teach a PR2 robot a generic task (pressing a button) under different task constraints. They taught the robot in two sessions on two different days to capture their teaching behaviour across sessions. The task performance was utilized to classify the provided demonstrations into high-quality and low-quality sets. The results show a significant Pearson correlation coefficient (R = 0.85, p < 0.0001) between the task performance and generalization performance across all participants. We also found that users clustered into two groups: Users who provided high-quality demonstrations from the first session, assigned to the fast-adapters group, and users who provided low-quality demonstrations in the first session and then improved with practice, assigned to the slow-adapters group. These results highlight the importance of quantifying demonstration quality, which can be indicative of the adaptation level of the user to the task